Lip synchronization of speech
نویسندگان
چکیده
Lip synchronization is the determination of the motion of the mouth and tongue during speech. It can be deduced from the speech signal without phonemic analysis, and irrespective of the content of the speech. Our method is based on the observation that the position of the mouth over a short interval of time can be correlated with the basic shape of the spectrum of the speech over that same interval. The spectrum is obtained from a Fast Fourier Transform (FFT) and treated like a discrete probability density function. Statistical measures called moments are used to describe the shape. For several canonical utterances, video measurements of a speaker's mouth are combined with the corresponding moments to produce continuous predictor surfaces for each of three mouth parameters: jaw position, horizontal opening between the lips, and vertical opening between the lips. The method involves smoothing so it is independent of the local behavior of the spectrum.
منابع مشابه
A Survey – Audio and Video Synchronization
The audio and video Synchronization is extremely necessary. The synchronization loss between image and sound continues to disturb observers and irritate telecasters. The demand is to assure synchronization without adjusting content at the same time as still retaining price low. The objective of the synchronization is to line up both the audio and video signals that are processed individually. T...
متن کاملAutomatic lip synchronization by speech signal analysis
In this paper a system for the automatic lip synchronization of virtual 3D human based only on the speech input is described. The speech signal is classified into viseme classes using neural networks. Visual representation of phonemes, visemes, defined in MPEG-4 FA, is used for face synthesis.
متن کاملAutomated Lip-Sync for 3D-Character Animation
A central task for animating computer generated characters is the synchronization of lip movements and speech signal. For real time synchronization high technical effort, which involves a face tracking system or data gloves, is needed to drive the expressions of the character. If the speech signal is already given, off-line synchronization is possible but the animator is left with a time consum...
متن کاملReal-time language independent lip synchronization method using a genetic algorithm
Lip synchronization is a method for the determination of the mouth and tongue motion during a speech. It is widely used in multimedia productions, and real time implementation is opening application possibilities in multimodal interfaces. We present an implementation of real time, language independent lip synchronization based on the classification of the speech signal, represented by MFCC vect...
متن کاملA Novel Analytical Approach for Lip Synchronization
We present a novel approach for Lip synchronization by analyzing the relationship between a person’s speech signal and data extracted from his/her lip movements. To model the speech we use a nonlinear-time-varying sum of AM-FM signals each of which models a single formant frequency. The model is then realized using Taylor series expansions such that a closed form formula is achieved which shows...
متن کاملPerformance Enhancement in Lip Synchronization Using MFCC Parameters
Many multimedia applications and entertainment industry products like games, cartoons and film dubbing require speech driven face animation and audio-video synchronization. Only Automatic Speech Recognition system (ASR) does not give good results in noisy environment. Audio Visual Speech Recognition system plays vital role in such harsh environment as it uses both – audio and visual – informati...
متن کامل